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Abstract 



Q 

. We introduce the concept of a deterministic walk. Confining our attention to 

the finite state case, we estabhsh hypotheses that ensure that the deterministic 
walk is transitive, and show that this property is in some sense robust. We also 
establish conditions that ensure the existence of asymptotic occupation times. 

^ '■ 1 Introduction 

cn 

Our purpose is to provide a foundational background for the study of deterministically 
O ' driven random walks on a state space 5". In particular, we are concerned to generalise 

properties of Markov chains to settings in which the dynamics governing the behaviour 
of the walk is deterministic, and where the Markov property does not necessarily hold. 
We define the deterministic walk as follows. 

, Definition 1.1 Let T be a measurable transformation of a probability space (X, m), 

and suppose that associated to each i E S is a. measurable function fi : X ^ S. We 
call each fi a transition function, and we call the collection {fk)kes an environment 
on S. 

Define the skew-product transformation Tf : X x S ^ X x S hy 

Tf{x,i):={Tx,fi{x)). (1.1) 
Define the deterministic walk on S in the environment {fk)kes to be 

Un:=Ui,n{x):=n2{T;;{x,i)) (1.2) 

where 7i2{x, y) := y. 
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An important feature of the above is that random updates are highly dependent 
on the current position of the walk. While there is a considerable wealth of dy- 
namical systems literature devoted to the study of systems in which transformations 
are applied randomly to a common state space, these predominantly deal with the 
situation where random updates are state-independent. A classic example of such 
systems is that of a random transformation i? of a probability space iY^v). Given 
a finite set T := {r^ : Y ^ Y : i = 1, . . . , X} of measurable transformations of 
a probability space iY^v), and an i.i.d. sequence {Tn)n>i of random variables tak- 
ing values in T, the random transformation R of (Y, u) is defined such that for all 
n> 1, i?" := r„ o . . . o Ti. An equivalent, deterministic, representation of R uses the 
skew-product Tf : x X — t- S+ x X such that Tf{uj,x) := {a{u),T^g{x)), where 
S := {1, . . . , K}, and a : S+ — t- S+ is the full-shift. In particular, for all n > 1, 

R^{x) = MT^^iu, x)) = r^„_, o . . . o r^„(x). (1.3) 

The ergodic theory of such systems was studied extensively in [B]. Among the many 
areas in which the model given by (11. 3p (and generalisations thereof) play an impor- 
tant role are the theory of products of random matrices (which goes back to the work 
of Furstenberg and Kesten |1]) and the theory of random iterated function systems 
(in which the properties of fractals generated by random applications of contractive 
maps to a complete metric space are studied - see [21 E] for early developments in this 
field). For a relatively recent study of position dependent case see 

While systems described by equation (II. Sp enjoy the Markov property, we are 
interested in understanding the asymptotic behaviour of deterministic walks that are 
non-Markov. 

Focusing on the finite state case, we establish recurrence properties under rela- 
tively mild assumptions. Notably, no mixing assumptions are required on the under- 
lying dynamics. We also note that while distortion properties play an important role 
in the infinite state case (see [8]), such assumptions are not required for establishing 
recurrence in the finite state case. 

In Section [21 we formally introduce the central notions of recurrence, transience 
and transitivity of a deterministic walk, and motivate these ideas by considering sim- 
ple examples. We then establish (in Theorem l2.8p conditions for which a deterministic 
walk on a finite state space is transitive, and show that this property is in some sense 
robust. 

In Section ini we establish (in Theorem 13. 5 p conditions under which a deterministic 
walk on a finite state space admits asymptotic occupation times. 
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2 Transitivity of a Deterministic Walk on a Finite 
State Space 



The following properties of a deterministic walk are fundamental to the exposition of 
our main results. 

Definition 2.1 Given a deterministic walk on a state space S in an environment 
(/fc)fcg5; we say that a state i G 5" is recurrent if 



Proposition 2.2 // S consists of two states, m{{x E X : fi{x) = i}) < 1 for 
each i E S, and T is ergodic and measure-preserving, then the deterministic walk is 
transitive on S. 

Proof Since m{{x G X : fi{x) = i}) < 1 for each i E S, and T is ergodic and 
measure preserving, it follows from Birkhoff 's Ergodic Theorem that m{r\^^QT~^ {x G 
^ • fii-f^) = i}) = 0. Hence for i,j G S, m{{x : Ui^n{x) = j for some n > 1}) = 1. I 

Example 2.3 In light of Proposition 12.21 it might be hoped that if S is finite, T is 
ergodic and measure preserving, and m({x G X : fi{x) = j}) > for all i,j G S, 
then {Un)n>o is necessarily transitive. As the following counter-example shows, this is 
not the case: Consider the deterministic walk on the state space S = {0, 1, 2} driven 
by the transformation T : [0, 1] — )■ [0, 1], where Tx := 2x mod 1, and with transition 
functions /o, fi and /2 given by 



m{{x G X : f/j,„(x) = i,foT some n > 1}) = 1. 



We say that a state i is transient if it is not recurrent. 

We say that the deterministic walk is transitive on S if for all i,j G S, 



m{{x G X : Ui^n{x) = J, for some n > 1}) = 1. 





and 
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Defining (3 := {[0, |), [|, 1)}, by inspection we see that the skew-product Tf : 
[0, 1] X — 7> [0, 1] X decomposes the product state space j3 x S into two closed 
communication classes given by {[0, |) x {1} , 1] x {0}} and {[0, |) x {0} , [|, 1] x 
{1}, [0,i)x{2}, [il]x{2}}. 

Since the first communication class does not project onto the whole fibre S, (in 
particular, the state 2 is not represented in this class) it follows that the deterministic 
walk is not transitive on S. In fact, it is easy to show that Tf induces a Markov 
chain on the state space x S (with respect to the product measure Lebcsguc x 
counting measure), and it follows that any walk started in the second communication 
class necessarily visits all three states infinitely often. 

Definition 2.4 Recall that a non-singular transformation T of a measure space 
{X, m) is said to be Markov, with a measurable Markov partition {3 of X, if 

(i) for all a e /5, T(a) is the union of elements of ^, 

(ii) T|a : a — > T(a) is a bijection. 

Given a Markov map T with with Markov partition /3, for all n > 1 and all sequences 
ao, . . . , ttn-i G /3, we call the set [ao, . . . , otn-i] '■= ^]=o'^ ''cij ^ cylinder set of rank n 
or an n- cylinder. 

For a given cylinder set a, wc let \a\ denote its rank. 

We denote by /3„ the collection of all n-cylinders. 

We say that a cylinder set a is admissible if m(a) > 0. 

We introduce concepts for Markov maps that are analogous to related concepts 
for Markov chains. 

Definition 2.5 Let T be a Markov transformation of a measure space (X, m) with 
Markov partition /3. Given a, 6 e /3 we say that a communicates with b if 

m{{x G a : T"x G b for some n > 1}) > 0. 

Given a, 6 G /3, we say that a and b intercommunicate if a communicates with b and 
b communicates with a. 

Given a E 13 we define the communication class of a to be the set of all 6 G /? with 
which it intercommunicates. (We observe that intercommunication is an equivalence 
relation.) 

We say that a communication class C is closed if for all a G C and all b E 13, a 
communicates with b, only if b communicates with a. (When /3 is finite, the existence 
of closed communication classes is automatic.) 

We say that (3 is irreducible if /3 is itself a communication class under T. 
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Standing Hypotheses for the Deterministic Walk We assume that a deter- 
ministic walk on a finite state space S is driven by a Markov transformation T of a 
probability space {X, m), with Markov partition /3. In proving the main result of this 
section (Theorem 12.81 below) it is convenient to assume that transition functions are 
constant on elements of (3, but as we observe later, this assumption is not entirely 
necessary 

Definition 2.6 Define the probability measure := ^ x counting measure. 

The following proposition is an easy consequence of the standing hypotheses. (The 
routine details are written out in [TJ Proposition 4.1].) 
Define := (3 x S. 

Proposition 2.7 The skew-product Tf : X x S ^ X x S is a Markov transformation 
of the probability space {X x S", /x) with Markov partition j3. 

Theorem 2.8 Suppose that the base map T is ergodic and invariant with finite 
Markov partition j3. Then the deterministic walk is transitive on S if and only if 
every closed communication class in /3 projects onto the whole of S. 

Proof By Proposition 12.71 Tf : X x S ^ X x S is a. Markov map with Markov 
partition /3. We observe that the 'only if direction of the proof is trivial, in that if 
there exists a closed communication class that does not project onto the whole of S 
then, clearly, the deterministic walk cannot be transitive. In proving the 'if part of 
the result we in fact prove something stronger. 

(i) Tf is transitive on closed communication classes in /3. 

(ii) If a partition element a x {i} G /3 does not lie in a closed communication class 
then fi{{y,j) e X x S : Tf{y,j) G a x {z} i.o.} = 0. 

Since S and /3 are finite, so is /3, and it follows from (i) and (ii) that with 
probability 1, an orbit under the skew-product dynamics must eventually end up in a 
closed communication class C C /3 on which the orbit is transitive thereafter. Hence, 
the deterministic walk is transitive on S provided that every closed communication 
class projects onto the whole of S. 

Proof of (i): Since T is Markov with Markov partition /?, and the environ- 
ment of transition functions {fi)i£s are constant on elements of it follows that for 
all n > 1, every admissible n-cylinder x := [xq, . . . , Xn-i], and all tq G S, there is a 
unique itinerary on S 

ro,ri, . . . ,r„_i 

such that fni^i) = rj+i for i = 0, . . . , n — 2. We thus define 

t{x,ro) := r„_i. 
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Fix a X {j},b x {k} G f3 such that both he in the same closed communication class 
C. Let 

a X {ji}, . . . ,a X {ji} k 6 x {zj, . . . , 6 x (2.1) 

be enumerations of all elements in C whose /3-coordinates are a and h respectively. 
Since T is ergodic and measure preserving, such collections are non-empty. We show 
that if we start in the communication class C then we must eventually visit h x {k}. 

Since h x {ii}, . . . ,h x all lie in the closed communication class C, then for 
each s = 1, . . . , n there exists a cylinder w{is) of the form [xi, X2, . . . , 6] such that 

t{\h,w{is)lis) := k, (2.2) 

and \b,w{is)\ is admissible (where in an abuse of notation we have written \h,w{is)] 
as shorthand for [6, xi, X2, • • • , &]). Equation (12.21) says that if the skew-product visits 
the set [fe, w(zs)] x {is}-, then it must eventually visit the partition element h x {k}. 

Since a x x {A;} G C, there exists an admissible cylinder Ei of the form 

[a, . . . , 6] such that 

t{Ei,ji) = k. 

For r = l, — 1 we may inductively define admissible cylinders 

Er+l = \Er,w{t{Er,3r+l))]- (2.3) 

For each r = 1, . . . , / the cylinder E^ is of the form [a, . . . , 6]. Thus, t{Er,jr+i) is the 
^'-coordinate at time \Er \ — 1 of every point in Er x {jr+i}- Hence, every point in 
Er X {jr+i} visits the partition element b x {t{Er, jr+i)} at time \Er\ — 1. It follows 
by construction that every point in the set Er+i x {jr+i} as defined in (12.31) will visit 
the partition element b x {k} at time |£^r+i| — 1- 

Since m{Ei) > it follows from Birkhoff 's Ergodic Theorem that the base dynam- 
ics must eventually visit Ei. Since we are in the closed communication class C, and 
since Ei C a, it follows that when the base dynamics visit the set Ei, the deterministic 
walk visits one of the states ji, . . . But since Ei C Ei_i C . . . C i?i it follows that 
we must eventually visit the partition element bx{k}. This completes the proof of (i). 

Proof of (ii): Fix a x {i} lying in a communication class C that is not closed, 
and let a X ii , . . . , a X be all the other partition elements in C whose /3-coordinate is 
a. Fix bx {k} G /3 with which a x {i} communicates, but which does not communicate 
with a X {i}. Let A denote the set of partition elements that intercommunicate with 
a X {i} whose /3-coordinate is b. Also, let A' denote set of partition elements whose 
/3-coordinate is b, with which a x {i} communicates, but that do not communicate 
with a X {i}. Clearly, b x {k} G A'. 

By a similar argument to that given in the proof of part (i), for all b x {j} G A 
there exists a cylinder w{j) = [xi,X2, ■ ■ ■ ,b] such that b x {t{[b,w{j)], j)} G A'. 
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Since a x {i} communicates with b x {k} there exists an admissible cyhnder Eq 
of the form [a, . . . ,b] such that 

t{Eo,i) = k. 

For r = 0, — 1 we may inductively define admissible cylinders 

^ f Er, if b X {t{Er, ir+l)} E A' 

\ [Er,w{t{Er,tr+imiibx{t{Er,tr+l)}eA. ^^'^^ 

Let a X {si },..., a X {st} be the set of all partition elements in 13 whose /3-coordinate 
is a, and with which a x {i} communicates, but which do not communicate with 
a X {i}. 

Assuming that the skew-product dynamics start in a x {i} then by Birkhoff's 
Ergodic Theorem, the base dynamics must (with probability 1) eventually make a 
first visit to the set Ei C a, at which point, the deterministic walk must be in one of 
the states . . . ,ii, si, . . . , St. If it is one of the Si (for i = 1, . . . ,t), then we cannot 
return to a x {i} by assumption. If instead it is in one of the states . . . , ii then, 
since Ei C Ei_i C . . . C Eq, by the same argument as in part (i), it must either 
eventually visit the state b x {k} or some other element of the set A'. By definition, 
it cannot return from any of these partition elements to the partition element a x {i}. 
This completes the proof of (ii). ■ 



Robustness of transitivity We call a set A G X transitive if for all i G S and 
all X E A, the orbit of the associated deterministic walk (f/j,„(x))^o visits every 
state in S at least once. Under the hypotheses of Theorem 12. 8[ it is immediate 
that if the deterministic walk is transitive on the state space S, then there must 
exist a transitive cylinder set a of positive measure. Any perturbation {f-)i^s of the 
environment {fi)i£s of transition functions that preserves the existence of a transitive 
positive measure subset a' G a will, by Birkhoff's Ergodic Theorem, preserve the 
transitivity of the deterministic walk. Therefore, we may say that the transitivity of 
the deterministic walk on a finite fibre is robust to perturbations of the environment. 
It follows that the hypothesis that transition functions are constant on elements of 
the Markov partition is not a necessary condition for the deterministic walk to be 
transitive on a finite state space. 



3 Asymptotic Occupation Times for a Determin- 
istic Walk on a Finite State Space 

In this section, we continue to assume that the standing hypotheses hold and we 
establish conditions for the existence of asymptotic occupation times of a deterministic 
walk on a finite state space. 
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Definition 3.1 Let T : X — )■ X be a Markov transformation of a probability space 
(X, m), with Markov partition j3. By the non- singularity of T, for all n > 1 and all 
a G we may define the Radon-Nikodym derivative v'^ := '^'■'""^ — ^ such that for 
every measurable set B, jrp„^^Q v'^dm = m{a fl T~'"-B). 

We say that T has the Strong Distortion Property, with distortion constant D > 1, 
if for all 77, > 1 and for all a G /?„, and for a.e. x,y E r"a, ^^7^ < D. 

The following proposition is an easy consequence of the standing hypotheses. (The 
routine details are written out in [71 Proposition 4.3].) 

Proposition 3.2 Suppose that the map T has the Strong Distortion Property. Then 
the skew-product Tf : X x S ^ X x S has the Strong Distortion Property with respect 
to the measure fi. 

Definition 3.3 We say that a Markov transformation T of probability space (X, m), 
with Markov partition (3, has Finite Images if H^{Ta : a G /3} < 00. 

The following result combines Lemma 4.4.1 and Theorem 4.6.3 in ^,J. (We note that 
[T] uses the term topologically transitive instead of irreducible.) 

Proposition 3.4 Let T he an irreducible Markov transformation of a probability 
space (X, m), with Markov partition j3, satisfying Strong Distortion and Finite Im- 
ages. Then there exists an ergodic, invariant, probability measure m' equivalent to 
m. 

We may now state the main result of this section. 

Theorem 3.5 Suppose that the state space S is finite and that the transformation 
T is a Markov map, with Markov partition P, that has Strong Distortion and Finite 
Images. Suppose also that /3 is irreducible under Tf. Then there exists a distribution 
i'^i)i€S such that for all i,jES 

^ n—l 

lim — > li(Ujr{x)) = TTj for m — a.e. x. (3-1) 

r=0 

Proof Since T has Finite Images and since S is finite, it follows that Tj also has 
Finite Images. 

By Proposition 13. 2^ Tj has the Strong Distortion Property with respect to 
By assumption, /? is irreducible under Tf, and it follows from Proposition 13. 4l that 
Tf has an ergodic, invariant, probability measure /i' equivalent to /x. Thus, if for each 
i G we define tt, := yu'(X x {i}), it follows from Birkhoff's Ergodic Theorem that 

^ n—l 

lim - li{Uj^r{x)) = TVi for //' - a.e. {x,j). (3.2) 

r=0 

Equation (13. ip now follows from (13. 2p . ■ 
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